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Abstract — Motivated by on-chip communication, a channel 
model is proposed where the variance of the additive noise 
depends on the weighted sum of the past channel input powers. 
For this channel, an expression for the capacity per unit cost is 
derived, and it is shown that the expression holds also in the 
presence of feedback. 

I. Introduction 

Continuous advancement in VLSI technologies has resulted 
in extremely small transistor sizes and highly complex mi- 
croprocessors. However, on-chip interconnects responsible for 
on-chip communication have been improved only moderately. 
This leads to the "paradox" that local information processing is 
done very efficiently, but communicating information between 
on-chip units is a major challenge. 

This work focuses on an emergent issue expected to chal- 
lenge circuit development in future technologies. Information 
communication and processing is associated with energy dis- 
sipation into heat which raises the temperature of the trans- 
mitter/receiver or processing devices; moreover, the intrinsic 
device noise level depends strongly and increasingly on the 
temperature. Therefore, the total physical structure can be 
modeled as a communication channel whose noise level is data 
dependent. We describe this mathematically in the following 
subsection. 

A. Channel Model 

We consider the communication system depicted in Fig- 
ure [U The message AI to be transmitted over the channel 
is assumed to be uniformly distributed over the set M. — 
{1,...,|A^|} for some positive integer \M.\. The encoder 
maps the message to the length-n sequence Xi,...,X„, 
where n is called the block-length. Thus, in the absence of 
feedback, the sequence X" is a function of the message M, 
i.e., Xi = (j)n{M) for some mapping (jy^ M. ^ M". Here, 



A" stands for 



,A„, and 



denotes the set of real 
numbers. If there is a feedback link, then X^, k ~ 1, . . . ,n, 
is a function of the message M and, additionally, of the past 

rk — l ;„ V _ ,_(fe)/)l/r -i^fe-l\ 



channel output symbols Yj^ 



i.e., Xk 



for some mapping ip\i' : A4 x R*^^^ M. The receiver 
guesses the transmitted message M based on the n channel 
output symbols F", i.e., M = ipn{Y{^) for some mapping 
V-n : K" ^ M. 

Let Z+ denote the set of positive integers. The channel 
output Yfc £ R at time k G Z+ corresponding to the channel 



yfe-i Delay 



M 



Xk 
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Transmitter Channel Receiver 

Fig. 1. The communication system. 

inputs (xi, . . . , Xk) G R'^ is given by 



Yk = Xk 



\ 



Uk 



(1) 



where {Uk} are independent and identically distributed (IID), 
zero-mean, unit-variance Gaussian random variables drawn 
independently of M. The coefficients {ai,} are non-negative 
and satisfjQ 



E 



a^, — a < CO. 



(2) 



Note that this channel is not stationary as the variance of the 
additive noise depends on the time-index k. 

We study the above channel under an average-power con- 
straint on the inputs, i.e.. 



fe=i 

and we define the signal-to-noise ratio (SNR) as 

SNR ^ 4- 

B. Capacity per Unit Cost 

Let the rate R (in nats per channel use) be defined as 



(3) 



(4) 



(5) 



where log(-) denotes the natural logarithm function. A rate is 
said to be achievable if there exists a sequence of mappings 
(j)n (without feedback) or Lpn^ , • • ■ , (with feedback) and 
such that the error probability Pr^M ^ tends to zero 
as n goes to infinity. The capacity C is the supremum of all 

'For on-chip communication the coefficients characterize the cool-down 
behavior of the chip, and it thus seems reasonable to assume that the 
coefficients are monotonically non-increasing, i.e., ai, < a^r for u > u' . 
This assumption is. however, not required for the results stated in this paper 



achievable rates. We denote by C(SNR) the capacity under 
the input constraint (O when there is no feedback, and we 
add the subscript "FB" to indicate that there is a feedback 
Hnk. Clearly, 

C(SNR) < Cfb(SNR) (6) 

as we can always ignore the feedback link. 

In this paper we study the capacities per unit cost which 
are defined as [1] 

Cfb(SNR) 



^ g(SNR) . 

^^^^ " ^^'P CMP ^"'l 

SNR>0 MNK 



Cfb(O) 



sup 

SNR>0 



SNR 



Note that (|6]l implies 



C. The Main Result 



C(0) <C'fb(0). 



(7) 



(8) 



Our main result is stated in the following theorem. 

Theorem 1: Consider the above channel model. Then, ir- 
respective of whether feedback is available or not, the corre- 
sponding capacity per unit cost is given by 

C(0) = (7FB(0) = i(l + a) (9) 

where a is defined in (|2]i. 

Theorem [1] is proved in Section |II] In Section |III] we 
briefly discuss the above channel at high SNR. Specifically, 
we present a sufficient and a necessary condition on the 
coefficients {a^} for capacity to be bounded in the SNR. 

II. Proof of Theorem[T] 

In Section III-AI we derive an upper bound on the feedback 
capacity Cfb(SNR), and in Section Hl-BI we derive a lower 
bound on the capacity C(SNR) in the absence of feedback. 
These bounds are then used in Section ITl-CI to derive an upper 
bound on Cfb (0) and a lower bound on (7(0), and it is shown 
that both bounds are equal to 1/2 • (1 + a). Together with ((Sjl 
this proves Theorem [T] 

A. Upper Bound 

As in [2, Sec. 8.12], the upper bound on Cfb (SNR) is based 
on Fano's inequality and on an upper bound on -^liM ', Y{^), 
which for our channel can be expressed, using the chain rule 
for mutual information, as 

n 

n 

k=l 
n 



fe=i 



1 " 

= -E 



k=l L 



/ fc-i 



1. 



log cr^ + E ^k~uXl 



v=l 



(10) 



where the second equality follows because Xi is a function 
of M and Y^^^; and the last equality follows from the 
behavior of differential entropy under translation and scaling 
[2, Thms. 9.6.3 & 9.6.4], and because Uk is independent of 
{Yt\M,Xl). 

Evaluating the differential entropy h{Uk) of a Gaus- 
sian random variable, and using the trivial lower bound 
E[log (a^ + T.tX^k-uXl ^ 
upper bound 

1 



> logiT^, we obtain the final 



-/(M;Fn 



< 



1 



fe=i 



)--log(27re(T') 



log l + ^afc_.E[X,2] /a' 




(11) 



where we define ao = 1- Here, the second inequality follows 
because conditioning cannot increase entropy and from the 
entropy maximizing property of Gaussian random variables [2, 
Thm. 9.6.5]; the next inequality follows by Jensen's inequal- 
ity; the following equality by rewriting the double sum; the 
subsequent inequality follows because the coefficients are non- 
negative which implies that X]"=o — X]^o o:^ — 1 + a; 
and the last inequality follows from the power constraint (O. 

B. Lower Bound 

As aforementioned, the above channel ([1]) is not stationary, 
and one therefore needs to exercise some care in relating the 
capacity C(SNR) in the absence of feedback to the quantity 



lim isup/(Xi";Yi") 



(12) 



(where the maximization is over all input distributions satis- 
fying the power constraint (O). In fact, it is prima facie not 
clear whether there is a coding theorem associated with ( fT2b . 
We shall sidestep this problem by studying the capacity of 
a different channel whose time-A: channel output Yk E R is, 
conditional on the sequence {Xk} — {xk}, given by 



\ 



fe-i 

E 



Uk 



(13) 



where {Uk} and {a^} are defined in Section ILAI This channel 
has the advantage that it is stationary & ergodic in the sense 



that when {X^} is a stationary & ergodic process then the pair 
{{Xk,Yk)} is jointly stationary & ergodic. It follows that if 
the sequences {Xk}k<o ™d {Xk}k>i are independent of each 
other, and if the random variables Xk, fc < 0, are bounded, 
then any rate that can be achieved over this new channel is 
also achievable over the original channel. Indeed, the original 
channel ([T]l can be converted into (T3[ by adding 



St 



\ 



— oo 



to the channel output Yk, and, since the independence of 
{Xk}k<() and {Xk}k>i ensures that the sequence {Sk} is 
independent of the message M, it follows that any rate 
achievable over (fT3] l can be achieved over ([U by using a 
receiver that generates {Sk} and guesses then M based on 

{Yk + Skjl^M 

We consider {Xk} that are block- wise IID in blocks of L 
symbols. Thus, denoting — (X^^+i, . . . , (where 
{■y denotes the transpose), {X^} are IID with X^ taking on 
the value 0, . . . , 0)"^ with probability S and (0, . . . , 0)"^ with 
probability 1—6, for some ^ £ R. Note that to satisfy the 
average-power constraint ^ we shall choose ^ and S so that 



L ■ SNR. 



(14) 



Let Ye = (YeL+i, • • ■ , and let [-J denote the 

floor function. Noting that the pair {(X^,Yf)} is jointly 
stationary & ergodic, it follows from [3] that the rate 



R= lim 



n V 



(15) 



is achievable over the new channel (fTJI i and, thus, yields a 
lower bound on the capacity C(SNR) of the original channel 
O. We lower bound i/(xL"/-^J"^ yL"/-^J-1) as 



L"/iJ-i 

ln/L\-l 



ln/L\-l 



-/ x:L;Y, 



(16) 



where we use the chain rule and that reducing observations 
cannot increase mutual information. By using that (|2]i implies 



lim >^ — Q 



it can be shown that the second term in the sum on the right- 
hand side (RHS) of ( fTSI l vanishes as £ tends to infinity. This 
together with a Cesaro type theorem [2, Thm. 4.2.3] yields 



lim i/(X, 



> i/(Xo;Yo 



xii 



[n/H-1 

■ioL[n/L\ E ^(XlL;Y, 



lim — 



L 



/(Xo;Yo 



(17) 



where the first inequality follows by the stationarity 
of {{Xe,Y()} which implies that /(X^ ; Y^IXi"^) does 
not depend on £, and by noting that, for a fixed L, 

We proceed to analyze /(Xq; YolXlj!^ — xlj!^) for a 
given sequence Xlj!^ = ^-L- Making use of the canonical 
decomposition of mutual information (e.g., [1, eq. (10)]), we 
have 



/ Xo;Yo 



X 



= /(^i;Yo 



dPx, (x) 



D 



(/yoI: 



(18) 



/yo|Xi=0,x:;^ 
/yo|Xi=0,xI^ 

/Yo|J>fi=0,x: 
/yo|Xi=0,x:;^) 

where the first equality follows because, for our choice of 
input distribution, X2 — . . . — Xl = and, hence, Xi 
conveys as much information about Yq as Xq. Here, 
denotes relative entropy, and /Yo|jfi=5,x:^' /yo|Xi=o.x:;^' 

™d /yoIxI 

puts (Xi = 

XlL = xl^, respectively. Thus, /yo|Xi=5,x:^ 
density of an L-variate Gaussian random vector of mean 
(^,0, . . . ,0)^ and of diagonal covariance matrix K^^ii with 
diagonal entries 



denote the densities of Yq conditional on the in 

iX_^ = (Xi — " V 1 



o,xi^ 



^-lo), and 



is the 



Kil (1,1)=-^ 



K^^l, {k,k) = (T'^ + ak-iC+ V a^tL+k-iXiL+i, 

i— — oo 

k^2,...,L; 

/yo|Xi=o x"^ density of an L-variate, zero-mean Gaus 

sian random vector of diagonal covariance matrix K 
diagonal entries 



-1 



-1 



.(0) 



with 



-The boundedness of the random variables X^., fc < 0, guarantees that the 
quantity Ylv^-x, '^k-u^t is finite for any realization of {Xfc}fe<o. 



Ki°2, {k,k) 



E 



a^iL+k-ix^L+i, 



1. 



-^Yolxi;^ is given by 



Yo|> 



Yo|Xi=5,x 



In order to evaluate the first term on the RHS of ( fTsT i we 
note that the relative entropy of two real, L-variate Gaussian 
random vectors of the respective means fii and /Xj and of the 
respective covariance matrices Ki and K2 is given by 

D(AA(/Xi,Ki)||AA(m2,K2)) 

= i logdet K2 - i logdet Ki + i tr (KiK^^ - 1^) 



(19) 



with dct A and tr (A) denoting the determinant and the trace 
of the matrix A, respectively, and where 1^ denotes the L x 
L identity matrix. The second term on the RHS of ( fTsT i is 
analyzed in the next subsection. 



Let E 



^(/yoIXI^ II/yo|Xi=0,X:1 ) 



denote the second 



term on the RHS of ( fTSl l averaged over X_;!^, i.e.. 



Ynix: 



/y 



Yo|Xi=0,X: 



Yolxl 



Yo|:ti=0,x: 



Then, using ( fT9l ) & ( fTSI ) and taking expectations over X_j^ 
we obtain, again defining ao ^ 1, 

i/(Xo;Yo|X:L) 



5 1 



EE 

/c=l 
L 



ak-1 



log! 1 + — 



> 



fc=2 



Yo|Xi=0,XI^ 

ak-1 



5 1 



L 



a. 



^^^log(l + a._ieV'x^) 



> 



-Z^K^^oix:^ 
^snrEt 



/y 



Yo|Xi=0,X: 



fc=i 



a • i • SNR 



^snrE 



log (1 + afe-i^VtJ^ 



fc=2 



/y 



Yo|Xi=0,X: 



(20) 



where the first inequality follows by the lower bound 
E[l/{1 + X)] > 1/(1 + E[X]) which is a consequence of 
Jensen's inequality applied to the convex function 1/(1 + x), 



a; > 0, and by the upper bound 



log 1 



< log (1 + ak-i^^a^) 



for every k ~ 2, . . . , L; and the second inequality follows by 
(O and by upper bounding 



for every = 1, . . . , L. 

The final lower bound follows now by ( |20] | and ( fTTI l 

lim i/fxL"/^J-^YL"/^J-^ 

n^oo 77, \ 



> 



1 

-snrV - 
2 ^1 



fe=i 



a • L • SNR 



log(l + afe_ieVf^') 



fc=2 



/y 



Yn|Ji:i=o,x" 



(21) 



C. Asymptotic Analysis 

We start with analyzing the upper bound ( fTTl i. We have 

CpB(SNR) ^ llog(l + (l + .).SNR) ^l 

SNR - SNR - 2^ ^ V ^ 

where the second inequality follows by upper bounding 
log(l + x) < X, X > 0, and we thus obtain 

CMO)^ sup ^i^<lil + a). (23) 

SNR>0 Z 

In order to derive a lower bound on C (0) we first note that 

. C7(SNR) C(SNR) 

C(0) = sup — — — — > lim — — — — (24) 

SNR>0 SNR - SNRiO SNR 

and proceed by analyzing the limiting ratio of the lower bound 
dlTT l to the SNR as the SNR tends to zero. 
To this end, we first shall show that 



lim ■ 

SNRiO 



YnlX- 



/yo|Xi=0,X:;^ 



SNR 



0. 



(25) 



It was shown in [1, p. 1023] that for any pair of densities /g 
and /i satisfying D{fi\\fo) < 00 



^^D{Ph + {l-P)M\fo) 



010 



0. 



(26) 



Thus, for any given X_i!^ = x_i!^, (|26] | together with 5 
SNR • L ■ crVC^ impUes that 



lim 

SNRiO 



^(/Volx- 


/yo|Xi=0,xI^) 


( 


iNR 



(27) 



In order to show that this also holds when 

^(/yo|x:^II/yo|Xi=o,x:^) is averaged over Xl^, we 
derive in the following the uniform upper bound 



sup D /, 



'Yolxl 



Yo|Xi=0,x: 



Yo|J>s:i=0,x: 



The claim ( 125] ) follows then by upper bounding 



.=0 



(28) 



D 



(/yo 



/y 



Yoi:ti=o,x" 



'Yo|x; 



Yo|Jfi=0,x: 



=0 



(29) 



and by ( |27l ). 

In order to prove ( |28] ) we use that any Gaussian random 
vector can be expressed as the sum of two independent 
Gaussian random vectors to write the channel output Yq as 



Yn — Xn 



V + W 



where, conditional on X*Loo = ^-oc ^ and W are L-variate, 
zero-mean Gaussian random vectors, drawn independently of 
each other, and having the respective diagonal covariance 
matrices Kv[xo and K^|^-i whose diagonal entries are given 
by 



Kv|x„(l,l) 

Kv|xo(fc7^) = o-^ + ak-ixi, = 2, . . . ,L, 



and 
K 



Wlx 



-1 {k,k) 



-1 

i— — oo 



Thus, V is the portion of the noise due to Xq, and W is the 
portion of the noise due to Xl;!,^. Note that Xq + V and W 
are independent of each other because Xq is, by construction, 
independent of Xl^!^. 

The upper bound (l28T l follows now by 



'Yolxl^ 



Yo|Xi=0,x: 



;, + V+W|x: 



/■ 



Xo+V+W|Xi=0,x: 



< D (/xo+v|| /xa+V|Xi=o) 



'Yolxli 



Yo|Xi=0,x: 



(31) 



where /x„+v+w|x:L /xo+v+w|Xi=o,xiL ^^^^ote the 
densities of Xq + V + W conditional on the inputs XlJ^ = 



and {Xi = 0,X: 



respectively; /xo+v 



denotes the unconditional density of Xo+V; and /xo+v|Xi=o 
denotes the density of Xq + V conditional on Xi = 0. Here, 
the inequality follows by the data processing inequality for 
relative entropy (see [2, Sec. 2.9]) and by noting that Xq + V 
is independent of Xl^!^. 



Returning to the analysis of (l2Tl i. we obtain from (l24l l and 



C(0) 



> lim - V — 

SNRI 2 1 



SNRio 2 ^ i + a- L- SNR 2 

k=l k=2 



^ i log(l + afe_ii5 

9 



k=l k=2 
C2 



(32) 



By letting first go to infinity while holding L fixed, and 
by letting then L go to infinity, we obtain the desired lower 
bound on the capacity per unit cost 



C(0)>i(l + a). 



(33) 



Thus, (I33, dH), and ^ yield 
1 



^{l + a)<C{Q)<C^{Q)<-{l + a) (34) 



(30) which proves Theorem [T] 



III. High SNR Results 



The channel described in Section II-AI was studied at high 
SNR in [4] where it was asked whether capacity is bounded 
or unbounded in the SNR. It was shown that the answer 
to this question depends highly on the decay rate of the 
coefficients {a^}. We summarize the main result of [4] in 
the next theorem. For a statement of this theorem in its full 
generality and for a proof thereof we refer to [4]. 

Theorem 2: Consider the channel model described in Sec- 
tion UlA] Then, 



i) lim 



> 



ii) 



lim 

f — >oo 



oo 0^1, 



sup Cfb(SNR) < oo, 

SNR>0 



= 



sup C(SNR) 

SNR>0 



OO, 



(35) 
(36) 



where we define, for any a>0, a/0 = oo and 0/0 = 0. 
For example, when {a^} is a geometric sequence, i.e., 
— /o" for < p < 1, then the capacity is bounded. Note 
that when neither the left-hand side (LHS) of ( l35T l nor the 
LHS of ( |36] | holds, i.e., when limjy^oo Q!i/+i/aj/ > and 
lim ,,^^ ttj^+i/a^ = 0, then the capacity can be bounded or 
unbounded. 

Acknowledgment 

Fruitful discussions with Ashish Khisti are gratefully ac- 
knowledged. 

References 

[1] S. Verdu, "On channel capacity per unit cost," IEEE Trans. Inform. 

Theory, vol. 36, pp. 1019-1030, Sept. 1990. 
[2] T. M. Cover and J. A. Thomas, Elements of Information Theory. John 

Wiley & Sons, 1991. 
[3] S. Verdii and T. S. Han, "A general formula for channel capacity," IEEE 

Trans. Inform. Theory, vol. 40, no. 4, pp. 1147-1157, July 1994. 
[4] T. Koch, A. Lapidoth, and P. P. Sotiriadis, "A hot channel," 2007, 

subm. to IEEE Inform. Theory Workshop (ITW), Lake Tahoe, CA, USA. 

[Online]. Available: http://arxiv.org/abs/cs/0703154 



